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ABSTRACT. 

Functional  occupation  measures  are  an  extension  to  occupation  measures 
on  llic  path  space  of  the  usual  definiiion  of  occupation  measures  for  stochas¬ 
tic  processes  They  are  used  to  get  limit  and  approximation  theorems  fc: 
average  cost  per  unit  lime  f>roblems  for  many  types  of  controlled  or  uricon- 
irm'i' d  randc'iii  (.[(.cesse,-  -Hi  tlwr  wtt  deabu /li;  mfiusion''  r'f:i 

difiusions.  and  singularly  perturbed  controlled  dmusions  There  are  exten¬ 
sions  to  wide  bandwidth  noise  driven  systems  and  to  man\'  other  models 
Tht  nieliiod  |>roMdes  a  convenicnl  and  powerful  wa\  of  ci.aracterizing  llit 
processes  associated  with  tlie  weak  limits  of  the  occupation  measures  and 
w;ti.  th'  sample  imnts  of  lli«  average  costs  per  unit  time,  as  tiie  various  pa¬ 
rameters  of  the  problem  to  tlieir  limits. The  melliod  can  be  used  tC'  get 
approximate  oplinialit}  tlieorems  and  similar  results  for  processes  which 
are  only  approximated  by  jump  diffusions  and  are  of  interest  over  a  long 
time  period, 


INTRODUCTION. 

In  this  paper  we  develop  a  powerful  tool  for  dealing  with  limit  problems 
and  approximations  for  controlled  or  uncontrolled  processes  which  are  of  in¬ 
terest  over  a  long  period  of  time.  Tnc  basic  applications  of  interest  concern 
ergodic  cost  problems  for  processes  governed  by  either  singularly  perturned 
st  ori.asti:  differentia!  equations  or  wide  band  noise  driven  systems  ;w  net;.e: 
singular!}  perturbed  or  noti  We  want  tc  s.now  that  certain  ‘averaged'  cr 
■'iim;t"  systems  car.  be  used  to  get  good  apprommations  to  the  optima. 
va!j‘.  functions  and  controls  for  tne  “physical"  systems  as  wcl.  as  to  cr.a-- 
acter.ze  the  “averaged"  systems.  Due  tc  space  limitations  we  concentrat' 
on  tne  singularly  perturbed  diffusion  mode!  and  on  reflected  diffusions  It 
will  be  apparan;  that  the  general  techniques  of  averaging  and  of  the  use  of 
the  '  functionai  occupation  measures"  which  art  employed  in  this  paper  art 
readny  usable  for  a  wide  variety  of  problems. 

The  fundamental  mathematical  technique  involves  a  way  of  character¬ 
izing  processes  associated  with  the  limits  of  certain  occupation  measures  in 
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a  way  that  is  mathematically  interesting  and  readily  usable.  The  occupa- 
tioii  measures  which  have  been  used  to  date  in  weak  convergence  analyses 
are  essentially  occupation  measures  over  the  state  space  of  the  process 
Tliey  measure  the  relative  amount  of  time  that  the  sample  paths  spend  in 
the  sets  on  the  state  space.  The  funcUonal  occupation  measures  are  occu¬ 
pation  measures  over  the  sets  in  iKe  path  space.  Their  main  use  here  is 
in  the  characterization  of  the  processes  which  >ield  the  pathwise  limits  of 
‘erpodic  averages”.  They  have  been  used  in  the  large  deviations  literature 
[12j,  but  in  very  different  ways  and  with  very  different  purposes.  In  this 
paf>er,  we  treat  the  background  ideas,  and  illustrate  some  of  the  results 
which  can  be  obtained. 

for  each  e,  h  i  t'(  )  denote  a  process  with  values  in  IT  .  Euciido.;,  r- 
spact.  and  let  (loosely  speaking)  t/'(  )  be  a  control  process  for  r'l  ).  For 
bounded  and  continuous  k{  ),  define  Yt  —  The 

z''(  )  might  be,  for  example,  a  singularly  perturbed  diffusion  or  a  wide  banc 
noi.se  driven  system.  In  the  singular  perturbation  case,  the  e  indexes  the 
singular  perturbation  In  the  wide  band  noise  case,  the  c  indexes  the  inverse 
of  the  bandwidth.  We  are  interested  in  the  limits  of  the  as  t  —  0  and 
as  7  —  oc,  and  in  the  approximation  of  t'(-)  by  ‘simpler  ”  processes 
r(  )  which  are  useful  for  approximating  the  values  of  functionals  of  t'(-) 
for  small  e  and  large  T.  These  simpler  processes  su-e  stationary  and  have 
mean  average  cost  per  unit  time  In  the  controlled  case,  we  want  controls 
that  are  ‘'good”  or  nearly  optimal  for  r(-)  to  be  ‘‘good”  or  nearly  optima! 
for  the  r'(  )  for  small  c  and  large  T.  The  functional  occupation  measure 
method  facilitates  the  characterization  of  these  simpler  processes  It  uses 
weak  convergence  techniques  connected  with  the  sequence  of  functional 
occupation  measures  for  the 

In  order  to  introduce  the  ideas  in  a  simple  wcy.  we  start  v.;tr.  the 
classical  occupation  mezisures  and  use  the  following  assumptions 

AJ.J  6f-),cr(  )  CTT  continuous  functions  with  o  grvwti.  ct  most  linear 
as  —  o: 


Let  u’(-)  denote  a  standard  vector- valued  Wiener  process.  Define  the 
process  x'  j  as  the  solution  to 

dr  =  fc(T) -i- c(  j)du  .  T  € /?' .  il-l  ) 

Let  A  denote  the  differential  generator  of  x(-). 

AJ.S.  {x(f).l  <  cve}  is  bounded  in  probability. 

A.j.S.  'l.l;  has  a  unique  invariant  measure 


If  the  slate  space  of  t{  )  is  not  bounded,  then  conditions  such  as  (A1.2) 
are  usually  verified  via  a  stochastic  Liapunov  function  method.  Let  P{S) 
denote  the  Borel  sets  of  the  metric  space  S.  Define  the  occupation  measure 

Q''(  )  K'' 

Q’(5)  = 5  €(/?*), 

and  the  normalized  occupation  measure 

Lf.  C'51  denote  the  continuous  real  valued  functions  on  5.  Ct(5)  the  subset 
of  hounded  funciioiis,  Co(5)  the  subset  of  functions  with  compact  support . 
and  Cl{R' )  the  further  subset  of  functions  whose  mixed  partial  derivatives 
arc  conlinuous-up  to  second  order,  when  S  —  R'  ■ 

It  IS  well  known  that  [1]  under  appropriate  conditions,  the  sequence 
of  measure-valued  random  variables  {Qt(  )}  converges  in  probability  (in 
the  weak  topology)  to  the  invariant  measure  (j[  ),  as  T  —  oc;  i.e.,  for  each 
/(•)  €  / f{x)Qr{dx)  —  / f{x)^i[dx)  in  probability  as  7  —  oc. 

Various  extensions  of  this  result  to  problems  in  stochastic  control  theory 
were  used  in  [  2]  to  prove  the  existence  of  an  optima!  feedback  control  for 
an  ergodic  cost  problem  for  a  controlled  version  of  (1.1).  They  were  used  ir. 
[3j  and  [4]  to  get  many  approximation  and  convergence  results  for  ergodic 
cost  problems  for  wide  band  noise  driven  systems  operating  over  a  ver\ 
long  lime  interval. 

In  this  paper,  we  take  a  point  of  view  which  is  more  general  and  which 
which  has  numerous  practical  advantages.  Let  Z?'[0.  oc)  denote  the  space 
of  /T-valued  functions  which  are  right  continuous  and  have  left  hand  limits 
and  which  is  endowed  with  the  Skorohod  lopoiog\  [o.  Chapter  o.o  ;  The 
space  Z)*[0.oc  )  is  complete  and  separable,  and  we  let  cv)  denote  its  gentri: 
element.  For  o(-)  in  D'[0.oc),  define  the  shifted  function  o, (■)  =  o(t  -  •). 
Define  the  shifted  process  ii(  )  =  x{i  —  -}.  and  define  the  occupation  meas,.:. 
P'{-)  by  =  ^{s,i  )}iBc)  Junctional  occupation  mtasurc 

Pt{Bo)  =  j  P-iBo)di.  (1,21 

Under  (A1.2),  {Pt{-),T  <  oc)  is  a  tight  sequence  of  measure-valued 
random  variables  (see  Sections  2  and  3  for  the  exact  definitions  and  proofs). 
Let  Pf{  )  denote  the  sample  value  of  Fr(-).  For  almost  all  u-.Pf{-)  is  a 
measure  on  5(Z?' [0.  oo)),  the  Borel  sets  of  Z?’'[0,oc),  and  hence  induces  a 
process  with  paths  in  £>’[C'.ac).  Let  P{-)  denote  a  measure-valued  ran¬ 
dom  variable  which  is  a  weak  limit  of  a  weakly  convergent  subsequence  of 
{Pt{-)..T  <  x).  Suppose  that  the  Pt(-)  and  the  P[  )  are  defined  on  the 
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same  sample  space  with  generic  variable  (*•.  Let  ]  denote  the  sample 
value  of  P(  ).  Then,  for  almost  all  u;,  the  measure  (■)  also  induces  a 
process  on  D'fO.oc).  It  turns  out  (Theorem  3.2)  that  this  process,  which 
we  write  as  is  a  stationary  diffusion  of  the  form  (1.1).  Since  u.  in¬ 

dexes  the  sampU  valut  oj  tht  measvn  P{-),  it  indexes  the  entire  process 
r"  (  ).  and  not  the  sample  values  of  that  process  The  necessary  probability 
background  appears  in  Section  2.  The  proofs  are  developed  in  Section  3 
We  spend  a  lot  of  time  on  the  simple  case  (1.1),  since  it  allows  a  relatively 
unencumbered  treatment  of  the  basic  ideas  of  the  functional  occupation 
measure  method.  Section  4  gives  the  details  of  an  application  to  a  control 
problem  for  a  controlled  form  of  (1.1).  Applications  to  the  singular!}  per¬ 
turbed  control  problem  on  the  infinite  time  interval  are  dealt  with  in  Section 
j.  Here  t)iC  P' (  )  are  for  a  singular)}  j.erturbed  controlled  dillusion.  and  v.t 
show  that  the  limit  of  the  samples  of  the  Pt(  )  induce  stationar}'  controlled 
diffusions  for  an  ’'averaged  system’".  Then  various  approximate  optimality 
theorems  arc  proved.  An  extension  of  the  results  of  Section  .3  to  a  reflected 
diffusion  apjiears  in  Section  C.  The  aim  is  to  outline  some  of  the  possibili¬ 
ties  Furiiier  applications  will  appear  in  [Cj. 

2.  PROBABILITY  PRELIMINARIES. 

Defiuitioiis.  Let  5  denote  a  metric  space  with  metric  d(-).  and  for  each  se: 
A  €  F(5).  define  the  set  A'  =  {x  :  d{x,y)  <  (  for  some  y  G  A).  Let  TiS) 
denote  the  collection  of  probability  measures  on  (5, 5(S)).  The  Prohorov 
metric  [5,p  96]  )  on  T{S)  is  defined  by 

r(P,  P')  =  inf{c  >  0  :P\A)  <  P(^’)  -  c, 
for  a!!  compact  A  € 

set  {Po(')}  €  PiS)  is  said  to  be  tight  if  for  each  f  >  C.  there  is 
a  compact  set  K{  €  5(5)  such  that  sup^,  Pc{t  ^  A',}  <  f.  Wt  say  that 
{Pr.(')}  tc  PiS)  contcrots  weakly  to  P(-)  in  P[S)  and  is  written  Pr.  P 
iff  P,./)  =  //(r)P,(dr)  -  (P./)  for  all  /(•)  €  0(5).  Vv'e  haw 

Theorem  2.1.  [4,pl0l].  7/(5,  d(  ))  is  complete  and  separable  then 
SC'  IS  the  metric  space  [P{S . 

Theorem  2.2.  [4.  pl04].  (Prohorov's  Theorem)  Let  S  be  complete  and 
separable.  Then  a  set  M  C  P(5)  is  relatively  compact  iS  ts  tight,  /./sc  . 
Pr,  zx-  P  IS  equivalent  to  r{P„.P)  —  0.  7/5  is  compact,  then  the  topology  of 
weak  convergence  is  equivalent  to  the  topology  under  the  Prohoroi  metric. 

Random  variables.  Let  A',,  and  A'  be  S-valued  random  variables  with 
associated  measures  Pr.{-)  and  P(-)i  resp.  We  use  the  terminology  tight¬ 
ness  of  {A'o}  and  weak  convergence  of  A\,  to  A'  (written  AV.  A')  inter- 
changably  with  the  terminlogy  tightness  of  {Po(-)}  weak  conver¬ 

gence  Pr.  =>  P.  The  sense  of  the  usage  of  the  words  tightness  and  weak 
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convcrgenct  depends  on  whether  the  objects  in  question  are  measures  or 
random  variable,,  and  the  abuse  of  notation  should  cause  no  problems.  The 
following  “Skorohod  representation"  will  be  very  useful 

Theorem  2.3.  [4,pl02).  Lei  S  be  separable,  and  lei  P«(  )  and  P(  ] 
be  iLe  measures  oj  the  S~valued  random  vartables  A',,  and  A’,  rtsj.  Lei 
^  A’.  Then  ihcre  exists  a  probability  space  (Q.P.T)  vilh  S~ralued 
random  variables  {A'„}  and  A‘  defined  or.  it,  such  that  A’,,  and  A’  have  the 
distnhutions  of  and  X,  resp.,  and  d[Xn.X)  —  0  u.p.J. 

We  will  use  the  Skorohod  representation  quite  frequently  since  it  sim¬ 
plifies  the  calculations  by  allowing  us  to  assume  that  the  weak  convergence 
IS  actually  w.p.l  convergence  in  the  appropriate  topology.  In  order  to  sim- 
p;i.*y  tilt  ijL'taiioii.  when  usiiig  the  Skorohod  repre.'t-ntat iciii  wt  w.il’  n:’ 
use  the  “hat"  symbol  for  the  new  sequence,  but  will  retain  whatever  the 
original  notation  was  for  the  sequence. 

Measure-  valued  random  variables  and  processes.  Let  St  be  a  complete 
and  separable  metric  space.  Then  V{Sd)  =  Si  is  a  complete  and  separable 
metric  space.  Let  Qr.  and  Q  be  Sj-valueo  random  variables.  Their  proba¬ 
bility  distributions  are,  of  course,  elements  of 'P(Si  ).  Suppose  that  {Cb  }  if" 
tight  (in  the  sense  of  a  sequence  of  random  variables)  and  let  Q,,  =?  Q.  If 
P  is  a  measure-valued  random  variable,  then  we  write  the  sample  value  as 
Q"  or  as  Q"  {■)■  Suppose  now  that  the  Skorohod  representation  is  used  sc 
we  can  assume  that  ■{(5„(-).n  <  oc,  (?(•)}  are  defined  on  the  same  sa.mple 
space  and  that  the  generic  variable  of  the  space  is  «-•.  By  the  weak  conver¬ 
gence  and  the  Skorohod  imbedding  for  almost  all  «,•  in  the 

topologv'  of  Si.  I.e., 

r(C:,C?“)-C  (2.1) 

for  almost  all  w.  For  each  w,  the  value  Q“(-)  is  a  measure  on  Sc  Let 
-.c  denote  the  canonical  point  of  So-  Then,  for  each  w-,  Q“(')  induces  ar. 
Sc-valued  random  variable  which  wt  denote  by  A’^  and  whose  values  are 
denoted  by  A'“(wc).  Note  that  the  w  and  n  index  the  meostre.  Wit;, 
the  measure  given,  wc  gives  the  sample  value  of  the  randorr.  ic^clle 

associated  with  that  measure. 

Now,  Theorem  2.2  and  (2.1)  imply  that  for  almost  all  w 


By  (2.2),  we  car.  use  the  Skorohod  representation  for  the  set  of  random 
variables  ({A'^  J.A'**)  for  each  fixed  u.  not  in  some  null  set  and  suppose 
that  they  are  defined  on  some  sample  space  with  generic  variable  -c  .  Then 
for  almost  all  w^c, 

(“'o)- (“'o))  ““  0.  (2.3) 

For  our  purposes  it  will  usually  be  sufficient  to  work  with  each  and  then 
the  relation  (2.o)  wiL  be  quite  useiul.  The  following  theorem  p>rcvides  a 
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useful  way  of  proving  that  a  sequence  of  T'(So)-valued  random  variables 
{Co  IIS'  light 

Theorem. 2. 4.  Svj>pos(  that  therr  arr  compact  Sr.  ]  Sc  such  that 

iimsupP{Ca(5c  -  5„)  > -}  =  0  (2.5) 

or.  cgufichritli 

limsupEQ^(So  -  S„)  =  0.  (2.C) 

^  D 

Then  iCo)  (OS  a  sequence  of  random  vanabltsj. 

Proof.  Gi\'  :.  f  >  C  .  need  to  find  a  compact  set  in  P(5;.)  su:ti 
that  supj,  P{Co  ^  T'i)  <  t.  By  (2.5)  for  each  c  >  0  there  is  a  compact  set 
5,  such  that 


sup  P{  Co  (Sc  -  S, )  >  f)  <  t. 

O 

Let  c,  —  0  and  ^  -  Given  b  >  0,  choose  such  that  c.  <  (■ 

Define  the  precompacl  set 

/w  =  {C  €  T(Sc)  :  C(St  -  S,.)  <  t.,f  >  m^). 

Since  supj,  P{Co  €  A'i)  <  the  proof  is  completed.  Q.E.D. 

3.  LIMITS  OF  OCCUPATION  MEASURES  FOR  DIFFUSIONS 

We  return  to  the  special  case  of  Section  1.  where  the  system  is  (1.1).  and 
Pt'.-)  is  defined  by  (1.2).  Define  5(  =  l?'[0.oc).  Theorem  3.1  givei  a 
criterion  f(o:  lii*.  tightness  of  the  normalized  occupation  measures 

Theorem  3.1.  Assume  (AJ.Jy  and  Then  Pj'-)  is  c 

valued  random  icricllt.  and  {Pr{-}.T  <  oc}  is  tight 

Proof.  For  each  B  t  P[Sc)  the  measurability  of  the  process  r, 
implies  the  .  f  l-measurabiiity  of  the  function  with  values  PG-. .  B )  Tnus 
Pj^-]  iJ  &  random  variable  for  each  T.  By  (.41.1)  and  (A1.2),  the  sequence 
of  processes  {z,(  ))  is  light.  Thus  for  each  t  >  C.  there  is  a  compact  set 
Kc  in  S[  such  that  P{x,(  )  €  5(  —  Ki]  <  f.  all  i.  This  implies  that 

EPj  {5c  -K,)  =  j  EP'iS'e  -  Ab  ]dt  <  A 

and  the  tightness  of  {Pi{-).T  <  oc}  follows  from  Theorem  2.4’  Q.E.D 

/.  remcrl:  on  notation.  Let  {Pj-^(-].n  <  oc)  he  a  weakly  convergent 
subsequence  witi,  limit  denoted  by  P-).  Let  .!■  be  the  generic  variable  or. 
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the  sample  space  on  which  j-(  )  is  defined.  Then  on  this  sample  space,  for 
each  w  ,  we  have  the  representation 


pTiG}=^J^  ))((7)di. 


(3.]n) 


(3.16) 


Now,  let  us  use  the  Skorohod  representation  for  <  oc,P(  )),  and 

let  the  generic  variable  of  the  common  sample  space  be  u.-.  Then,  using  the 
same  notation  {/’T«(')-ri  <  c5C.P{  ))for  the  imbedded  random  variables, 
1  0  1  b  I  continues  ic  hold  in  that  the  distributions  of  liie-  twc'  sides  are  e-tiiia; 
for  each  T  and  C.  Let  denote  equalit}'  in  the  sense  of  distribution 

Theorem  3.2  Assvme  (ALS)  and  lei  Tr.  —  cx:  tndex  o  v'takli: 

convergent  subsequence  of  {Pj-(-),T  <  oo)  uilA  liTn:(  denoted  by  P{  ). 

Assume  that  the  Skorohod  representation  ts  used  and  that  w  denotes 
the  generic  variable  of  the  sample  space  on  which  {Pr,.  (  l  *^  <  ^■■P(  ); 
defined.  Then,  Jor  almost  allu.,  the  sample  value  P“(-)  of  P(  )  induces  a 
stationary  diffusion  process  r“(-)  on  So  with  differential  operator  A.  Let 
p'*!'  )  denote  the  stationary  measure  o/z‘^(  )  .  Let  A'(  )  denote  a  bounded 
and  continuous  real  valued  function  on  Sq.  Then 

^  K(z,{.))di=^  J  /•;(c>(  ))P(dc.) 


=  rA'(P'(-)). 


For  A-  .  )  C  Ci'P’ 


Y-  j  k[z{s)tds  =i-  j  klxfLidz),  (o.c  ' 

whe^  L,-)  IS  the  measure  valued  fon  the  Bortl  sets  of  P'  .;  random  venal'.: 
wit't.  values  /’’'(■I  Under  (Al.Sj.  jor  almost  allu. 

P"  f)  =  t<()- 

OTid  the  subsequence  is  irrelevent. 

Proof.  Staticnarity.  We  use  the  Skorohod  representation  for  {Pr„(')' 
n  <  oc.Pi  )}  all  through  the  proof.  Let  G  €  S^,  and  define  its  left  shift 
Cc  =  {<?-)  ;  c-aO  €  G).  Then 

Pt[G)=^-  p.(Gc)*  =  -  /  )t(G)dt 


htc.)-  „(G)*  -  i  (C)JL 

Ttius  Py  (G)  —  Pj^  (Gc )  —  C  as  T  — •  0  for  all  u.  and  G.  This  and  the  weak 
convergence  implies  that  P“(G)  =  P“(G.)  for  all  w  and  all  sets  G  with 
P"  i  i^G )  =  0  Tins,  in  turn,  implies  the  stationarity. 

li.  the  rest  of  the  proof, we  will  only  characterize  the  process  induced 
by  the  values  P**(  )  of  the  Umit  P(  ).  By  the  definition  of  the  sample 
occupancy  measure  Pj  {■).  for  each  real-valued,  bounded  and  measurable 
function  Pc(  )  on  Sq,  we  have 


I  Fc{x,{-))d1  (3  ) 

Ir.  fact,  (3  4)  completely  determines  Pt(-)- 

Let  j  tie  a  bounded  and  continuous  real  valued  function  of  its'  ar¬ 
guments,  let  9  be  an  arbitrary  integer,  and  let  t,.r.r  -f  s  be  such  ttiat 
—  s.:  <  <;  Let  /(•)  £  Cq(R').  Define  the  function  P(  )  b\ 


r-*-' 

F(c(-))  =  <  9)[/(<!>(r-f-s))-/(d>(r))-y  A/(o(u))c/u:. 


(3.5) 


Tiie  function  F(-)  is  measurable  but  it  is  not  continuous  at  all  points  c-^  j. 
It  is  continuous  at  each  <?(•)  which  is  continuous  [6,  pl21j. 

Since  the  processes  induced  on  S[,  by  the  Pj  (■)  are  continuous  for 
each  w  and  T.  the  sample  values  P‘'(')  of  the  limit  measures  alsc  have 
the;*  support  on  the  set  of  continuous  functions  in  S^,  Hence,  even  though 
the  function  P(  )  is  not  continuous  at  every  point  in  S[,.  it  is  continuous 
V.  *  ,1  relative  tc  P“  '•)  for  almost  ah  .  since  it  is  continuous  at  each  po.ni 
0  -  I  which  IS  a  continuous  function.  Due  tc  this  fact  and  the  Skorohod 
representation  [7,  po.lj. 

I"  F(C)P^Jco)  —  I  P(c.)P'' (a’oj.  (S.C! 

fo:  almost  ah  Let  E~  and  P'(-)  denote  the  expectation  with  respect  tc 
P".  )  and  the  process  which  is  induced  by  P''(-).  resp.  The  result  (3.2)  is 
just  a  consequence  of  the  weak  convergence.  By  the  definition  of  P(-).  the 
rign:  hand  side  of  (3.4),  evaluated  at  w  equals 


P^o)P‘-(do)  =  £^/i(^’'(i.)i>  <  <?)• 

:/(f-  (7  -  s))  -  /(f^V))  -  ^  //>-•*  iv}  ,dv: 
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It  will  be  showr*  below  that  (3.7)  equale  lero  for  almost  all  u.  Thus,  owing 
to  the  arbitrariness  of  ,/{■),  for  almost  all  w  the  process 

i-“(  )  must  solve  the  martingale  problem  for  the  operator  A. 

Let  =  lr(C'(0)).  Then,  for  almost  all  u>,  A’(  )  is  continuous 

VI  p  1  relative  to  the  measures  P“(-).  Note  that 

^  f^i^s{  )ds=^  J  K{<p)PT{dc>), 

I  A(c)/-'^ta’c-)  -  J  A'(c-)/’“(tfo)  =  J  t(c<(0)P“(do(0)). 

!•>  till  statioriarii }•  of  for  almost  all  w,  this  la.'t  cxjinssioi.  or. 

till  riptit  equals  (for  almost  all  «.)  j  k{T)fi'‘  [dx). 

Proof  that  (3.  7y  tqxiah  zerci  for  almost  allu,\  B\  lie's  Formula  and  the 
represeniation  (3. 4),  we  can  write 


I  Fic]P^[dc^)  =  ^ 


9{v)dvii) 


for  some  bounded  and  non-anticipative  p(-)  This  expression  goes  to  zero 
in  probability  as  T  —  oc.  This  fact  together  with  the  weak  convergence 
implies  that  (3.7)  equals  zero  for  almost  all  w.  Q.E.D. 


4.  THE  CONTFtOL  PROBLEM. 

Definitions  In  order  to  present  tne  results  in  a  relatively  simple  wav  v.c 
work  with  stochastic  relaxed  controls  [3'.  [8].  Let  the  space  of  values 
for  tne  controi.  be  a  compact  set  ir.  some  Euclidean  space.  Let  7-  de¬ 
note  a  filtration  and  u  (•)  a  standard  vector-valued  fF, -Wiener  process.  Let 
Mi{V  X  [0.c>c))  =  So  denote  the  space  of  measures  !'(•)  on  BPf  x  [C.ocU 
with  the  following  property:  i/{l'  x  [C.ij  =  I'i’C-f)  =  i.  ali  :.  Such  i-'.'-) 
are  called  (deterministic  )  relaxed  controls.  On  So  we  put  the  ‘'compact- 
weak''  topology:  i.e..  i/,. (•)  —  i/(-)  iff  [Q-I't.)  —  (o.!/)  for  each  <?(■)  in 
Co{l'  X  [0.  oc)j.  Under  this  topology,  S'f  is  a  complete  and  separable  metric 
space.  If  !/(•)  €  Sc',  then  there  is  a  denvatirt  i/, (•)  such  that  i/j(-)  is  a 
measure  or.  5(1')  for  each  i.  i'j(C)  is  t-measurable  for  each  C  G  B(r).  and 
i/{dadi)  =  i/i{dQ)di.  Define  So  =  So  x  S'f  ■ 

At,  admissible  stochastic  relaxed  control  rn(-)  is  an  Sc'-valued  random 
variable  such  that  tvfC.i)  is  T.  — adapted  and  m(C.  •)  is  measurable  for 
each  C  €  5(i')-  The  derivative  m,(-)  car.  be  constructed  so  that  for  each 
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C  €  P[C').  rn,(C)  IS  -adapted  If  u(  )  is  ar.  f,-  Wiener  process,  we 
sometimes  say  that  (r7i(  ).u(  ))  is  admissible  or  that  m(  )  is  admissible 
with  respect  t:  ii(  ). 

For  admissible  ("i;  ), ti{  ))  the  controlled  SDE  is  wriiien  as 


dr  = 


I 


fc.'r.Q)m,(da  )di  ■+  £r(j-)cj’u  , 


(4.1) 


We  use  the  following  replacement  for  (A  1.1). 

A4.1  <t(  )  and  6(  )  arr  continuous,  er(  )  and  1>(-,q)  have  at  most  a  linear 
grovih  as  ;j'  —  oc,  untformlt  in  q. 


Let  A°  dcnr.ie  tiie  differential  generator  of  (4.1)  with  liie  coiilrc',  f;.\-c 
at  Ci  G  f.  For  an  admissible  stochastic  relaxed  control  m(  ).  let  A^'  denote 
the  differential  generator  of  t(-).  Then,  acting  on  smooth  functions  of  x  at 
lime  i.  we  can  write  A^f(x)[i)  =  J  A°  f[x)in,{da).  Define  the  'shifted’' 
stochastic  relaxed  control  A,m(  )  =  m[i  -f  •)  —  rn{i).  The  relaxed  control 
A;m(  )  is  tlie  one  that  is  actually  used  on  the  shifted  process  x.(  )  since 
(A;f7i),(  )  =  and 

dx,{f)  =  ds  J  b(x,{s).o)m,^,{dQ) ->r  er{x,{s))dv.,{f). 

Dehne  the  occupation  measures  P"' {■)  and  Pt{-)  by 

FiBr  X  Cc)  =  )}(Cc). 

Here.  Cc  €  and  Be  €  5{5t.}. 

Tlifcorem  4.1.  Assume  {AJ.Sj  end  (A.*.J).  Then  {Pj-{-).~  <  or]  is 
c  tight  set  of  T{S:  ^-veIued  random  variables.  Let  P[-)  denote  the  limit  of 
some  ueakh  convergent  substguence  (indexed  bv  Tr.  —  oc^,.  end  suvvcsi 
that  the  Skorohod  representation  is  used  /or  {/j-^  (•).  n  <  oc.P7(  j}.  Then, 
for  almost  allu:,  /’'"(•)  induces  a  process  (f"  (•),  nt’’ (•  j)  on  Sr  end  the  dis¬ 
tribution  of  the  shifted  pair  (z,*"  (•).  Ajm*' (■))  does  not  depend  on  i.  For 
each  w.  there  is  a  filtration  Tf  and  an  Tf  -standard  vector-valued  U'lcner 
process  u‘'(-).  such  that  fh**/-)  is  admissible  (with  respect  to  w'‘y)J  and 

dx''  =  J  b[x'"  ,Q)ihj  [dQ)dt -r  er^x"  )du'‘ . 

Define  the  random  variable  K  with  values 
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rj:i^ 


(z{i).Q)m^{dQ)di  ^  K 


Proof.  Tlie  proof  is  similar  lo  that  of  Theorem  3.2.  The  set  of  pro¬ 
cesses  {A,  m(  ),<  <  oc}  is  tight  and  so  is  the  set  {i,  (•).l  <  oc).  This 
implies  the  tightness  of  {/7-(  ).r  <  oc),  and  we  need  only  characterize  the 
limits  of  the  weakly  convergent  subsequences  and  explain  (4.2)_  Let  T„ 
index  a  wc  kly  convergent  subsequence,  with  limit  denoted  by  Pi  )-  Let 
(f*  (•).rTi‘*  (•))  denote  the  process  induced  by  P‘*(  ).  The  asserted  station- 
anti  IS  proved  as  ir.  Theorem  3.2.  Let  V’(  )  and  t_,(  )  be  in  Coil  ^  [C 
and  define  ft'.i'lt  =  Q)i/(dod.«)  =  i’(f.  o)t',ido)df .  Ci  'ose 

/i')- V- b  "c  as  in  Theorem  3.2  and  lei  ;>  be  an  arbitrari  inngv; 
Define  the  function  F(  )  on  Sc  by 

/'(c-.i')  =  <  gj  <  0)  -  fici'})- 


ri 


fioi" )  )i'{dadu )_ . 


Bv  Ito's  Formula  and  the  definition  of  Pt{')> 


Fic,  u)PTidc‘ 


F iT,i-).  A, m{-))di  = 


-  j  /i(z,(i.),(V;.A,m),..t  <  5,j  <  r)cfi  y  /;(i,(u))r(i,(u))du.^l 

Let  us  use  the  Skorohod  representation  By  the  same  argument  used  ir. 
Theorem  3.2.  we  have,  for  almost  all 

I  P{C’.  t')P(dpdi')  =  0.  (4,4i 

Equivalently,  for  almost  all  w.  and  with  £“  denoting  the  expectation  w.th 
respect  tc  P"  {■). 

r- A(f-  (;, ),  (t ..  <  g.j  < 

J  J  >=‘’/(f''(u))m’*(dQdu)]  =  0.  w.p.l. 

Analogous  to  the  situation  in  Theorem  3.2.  the  process  solves  the 

martingale  problem  with  operator  A**"  and  with  respect  tc  the  filtration 
(s).  m*"  (•.  s).  s  <  <).  All  the  conclusions  except  the  representation  on 
the  right  hand  side  of  (4.2)  follows  from  this. 
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Define  the  functjor.  A'(  )  on  Sc  by 


Kio.i') 


-fj 

Jc  Jv 


ki0{f).Q)i/(dsdQ). 


The  right  hand  side  of  (4  2)  is  obtained  by  noting  that  the  limit  of  the  left 
hand  side  of  >,4  2)  equais  the  limit  of 

^  ^  K[x,{  ).Xmi  ))di=^  J  K{d>,i')PT{dpdi^). 

Q.E.D. 

SINGULARLY  PERTURBED  CONTROL  PROBLEMS: 
LIMIT  AND  APPROXIMATION  THEOREMS 

It;  tins  section,  we  work  with  ilie  siiigularK  |>erturbed  diffusion  model 

dx’  =  '  I  C{x' .  z'  .a)m[{dQ)]di  +  (t(j' ,  r')du  • ,  (5  1) 

(dz’  =  H{x'  ,z')dt  +  \/7v{x\z')du‘2,  (o.2j 

o(a-,r)  =  £r(i,r)r'(r,r). 

The  u, (•)  arc  standard  vector  valued  Wiener  processes  with  respect  to  some 
filtration  T, .  and  m'(-l  is  an  admissible  relaxed  control  (always  with  respect 
to  or  (ti ; {•)•  ti>;(-))).  The  control  takes  values  in  a  compact  set  V.  The 
model  (5.1).  (5.2).  is  the  most  common  one  used  for  stochastic  control 
problems  unaer  singular  perturbations  [6]. [Pi. [10*  The  mode)  will  aliow  a 
relatively  simple  and  generic  development  of  the  averaging  methods 

Dcfinittci,  Wc  say  tha:  the  solution  to  (5.1).  (5.2)  is  triftf  it.  t:.i 
uicl:  .‘fT;«c  if  the  d'strioution  of  (m'(-).  u  ;(■).  u  ;,(■))  determines  inai  of 

;r';). TT;'(-).U’;(-).ti2(-)) 

We  use  the  analogous  definition  in  the  absence  of  a  control  ^Ve  use 
Fr,  to  denote  the  e.xpectation  under  the  control  m'(-).  and  with  the  initial 
ccnditior.  fr.rt  Toe  distribution  of  (z'(-),  r'(-) )  actually  depends  on  the 
joint  distribution  of  (Ui(  ).ti2(  )  ’^^(■))'  but  we  omit  the  Wiene:  processes 
from  some  of  the  notation  for  simplicity. 

Tin  fixtd-x.  rtscaltd  'fast  process’’.  We  exploit  the  time  scale  differ¬ 
ences  between  r’(  )  and  :'{■)  in  order  to  approximate  (5.1)  by  a  simpler  ‘’av- 
eraged”  system  Define  t’ne  ‘‘stretched  out’’  processes  rQ(f)  =  = 

x'l'ci).  Then 

c'-'f  =  I  Q)rr/J^dcj):dt  -r  zl^du:  (5.3) 
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dzl  =  J^(xlzl)di  +  tizlzl)dv-j.  (5,4) 

wliere  llie  il,(  )  are  standard  Wiener  processes  In  fact.  u,(t)  =  Ui{d)/v^ 
Since  Xq(  )  IS  ‘‘nearly  constant’'  over  “lonp*^  time  intervals  for  small  e,  (5.4) 
suggests  that,  on  the  appropriate  long  time  interval,  we  can  treat  rp(  )  as 
though  Tq{  )  were  fLxed,  and  use  this  fixed-a-  process  to  average  out  the 
the  r'(  ).  Wit)i  this  in  mind,  we  define  the  fiiid-x  jirvciss  rel  ief)  (written 
simply  as  20 (  )  if  t)ie  parameter  is  known)  by 


dzi,  =  H(t,zo)  +  r(T.2o)dui.  (5.5) 

L(i  Ag  dtnoit  tLi  differential  generator  of  the  fixed-x  prveesf^  with  pa- 
rciii(t<r  r.  A'  the  diffe  re  nl'i.l  generator  of  c’l,  ) ).  arid  A^''  that  (.f 

I  2(. (  il  We  will  use  tilt  following  assumption 

A5  l./or  each  initial  condition  and  each  x,  (5-5)  has  a  unique  weak 
sense  seduticn  for  each  i.  ip(-iT)  has  0  unique  inianant  rncasurx  /i.f  ). 
There  is  c  continuous  matrix  r(  )  such  that 

r^x)r'tx)  =  Q(a-}  =  /  C(X.  :  }r'(x.  z  )fXj  (dz ). 


The  factorizatior.  assumption  or.  c(  )  is  a  convenience  for  the  notation, 
since  it  allows  a  simple  and  concrete  representation  of  the  averaged  process, 
but  it  IS  not  necessary  for  the  validity  of  the  basic  results. 

The  ai'craced  sgsiem.  Define  the  averaged  functions  and  system  by 

G^x  a>=  I  G  x.z.atfz.ldz).  k[x.Q)  =  J  k{x .  z .  Q)ug(  dz  ) . 

r 

dx  =  dt  I  C  x.Q  rr,.  '.do  1’  —  r.  j  tdu  1;  C  ■ 

Here  m '■)  is  an  admissible  reiaxec  contrcl  with  respect  tc  the  standard 
vectct-va.uec  Wiener  process  w  1 

Let  A°  denote  the  differential  cenerator  of  (5  C.  with  cor t’X'l  fixed  at 
a  and  let  A'"'  he  that  associated  with  the  relaxed  contrc’l  m'  ]  We  wd;  use 
the  fcliov. nng  assumptions 

/.i  1  G.x.z.o)  =  Gd',x.x)  -  Gy  x.a  ) 

k,X.Z.O}x:  kt'.X.z)-  k-yX.Q). 

The  c\  )  and  the  G,-  1  are  continuous  and  have  at  most  a  linear  grvuth  in 
X  as  X  —  oc  uniformh  in  o.r  The  k,{  )  ere  bounded  and  continuous 

A5.3  H[-j  and  t  )  ort  continuous  and  have  at  most  a  linear  growth 
in  z.  unifor-ntli  in  x 


Vc 

% 


Ocir)  =  j  Go{z.2)^i,{d:).  ko{x)  =  j  kc{z,:)^Itid:). 
The  cost  fvneUen.  Define  the  pathwise  and  mean  costs: 

->f  (r.r.m'.t'i.u’i)  =  ^  j  k{z’ (s).  r' (s).  Q)m[{dQ  )ds , 
r,  m',  u'l,  U'2)  =  limsupf-)j(x,i,m'.tii,Ui), 


inf  •)' [x .  :  ui' .  V .v  r) 
adni  controi' 


■)T{x,Tn.w)  =  —  J  J  k{x[s).Q)m[{dQ}di. 

7(2-.  m,  u  )  =  limsup  £'77-(i,  m.w),  (5  S) 

T 

7(,(2)  =  inf  7(2, m.u). 
adm.  controls 

We  keep  the  arguments  in  •)'  since  the  value  dej-ends  on  the 

join;  distribution  of  the  triple  and  not  just  on  m' .  I:  the  initial 

condition  2  is  omitted  in  'tc{z)  or  in  7(2,  m,u  ),  then  the  values  do  no; 
depend  on  2. 

Note  that  neither  7j(-)  nor  77  involve  expectations.  Ttiey  are  sim¬ 
ply  pathwise  averages.  The  main  results  are  Theorems  51  and  5.2  below 
Loosely  speaking,  w’e  will  show  that  as  c  —  0  and  J  —  00.  the  patnwist 
average  cost  (2.  i.  m' .  u ; .  u 2)  will  converge  to  the  mean  cost  per  unit 
lime  for  an  averaged  problem,  where  the  system  is  (5.6)  and  the  cos;  is 
*>  '2.  m  u  ;.  Also,  under  broad  conditions,  given  ^  >  C  there  will  oe  some 
sequence  {m'(-)}  such  that  7^ (2.  r.  m' .  u; .  U2)  converges  in  probabihtj  ic 
within  f  of  the  infimum  7c.  of  the  costs  for  the  averaged  problem,  and  there 
IS  no  admissible  {rTj'(-)}  for  which  the  limit  is  less  than  Tne  wa\  tha: 
<  —  0  and  T  —  oc  wih  no;  be  important.  Such  results  help  to  just;:';, 
use  of  the  averaged  problem  even  on  the  infinite  time  interval  \\t  wih  use 
the  condition 

A5  4./f'’'  cccK  {  >  Q  and  c  >  0,  there  is  0  f—Dj’timcl  (fo-r  jf.e  cc-r 
fvncticnc!  {x .  : .  rr,'  .u-^  .U2)J  admiifit  U  m'(-')  fx.ch  that  the  ccrrespondinc 
stchtnct 


{-'(O- -'(•)•*  <0:0  C] 


The  condiiion  (A5.4)  implies  that  there  are  optima!  policies  for 
which  the  system?  a'(-))  do  not  explode  as  t  —»  oc  and  e  0 

We  follow  the  terminology  of  Section  4.  Define  the  functional  occu* 
fianiion  measures  /*'•’(  )  *snd  /*f(  )  hy 

Theorem  I».l.  Asst/me  (A5.1J  tc  {A5.4J-  rtlaitd  controh 

■:  f  Vi  fcr  scinf  /  >  C  Thfv.  lK(  ffgvfncf  r/ T(5c)-»’n/iifd  random 
lariailts  )  i  >  O.T  <  oc  ]  is  Iiphf  let  /\)  d'voti  a  v(ak  limil  ,  /  ^ 

sofiK  stgtifuct  ir.  —  0,  T„  —  oc^  vith  %atuc^  P“(  )•  For  almost  all  ^ 
/*“  )  vtdvcfs  c  stationery  prnrtss  ( j*"  (  )  or.  Sr  tr  iho  srrisf  il.ct 

tKr  diftntivtiori  n/ (zj*  (•),  does  not  depend  oni.  For  almost  ollu., 

li.trr  i‘  c  standard  i  ector~t  alued  M’lffirr  process  n‘'()  svcl.  that  tti“  (  i  i‘ 
cdmissii.U  vitl.  rcsj’CCt  to  u-v)  and  ir“{  )  rh“(  )j  sctish  (i  ( 

r“  J  J  t(a“  (f) 

Remark.  In  the  last  equation  is  the  cost  for  the  staliona*;. 

prooierr.  where  I'fit  initial  condition  f‘‘(0)  is  the  random  variable  w  itt.  i:.i 
stationary  distribution 

Proof.  Tn?  {r',  |  t  <  oc.e  >  0}  is  tight  due  to  (.4b. 2i  and  (Ab4i  Tne 
St;  {2^.,n.\  )  t  <  o..(  >  C)  IS  always  t;g;-,t  Tnese  facts  imp:\  t:.e  ■AS^cf.^z 
t.g.'.;r.e.‘?  c'  {F~  }.(  >  i  .T  <  oc) .  analogously  tc  the  case  of  Tm  ore.t.  c 

Let  F  ■).  ti  ,  )  i,  X.  i.p  q .  r.  s  be  as  in  Theorem  4.1,  and  ie:  denote  t.ne 

e.vpectatior.  v,  ,::  respect  ic  Fy  “  L;  Tut:.  ar.aiogcus:\  tc  i;,<  ca.  . 
ir.  Theorem  4  1.  by  he's  Formula  and  the  definition  of  Pj[  )  we  have 


I  ' 

J  j,  ^  9- J  <  -r  I 

-ffx'fr))-  I  du  j  /;(i;(u))G(x|(u).c;(ti),Q)(A,T7i'jJdo 

1  \ 

--  I  du  trace /,.ix;(u))  ■ 


lb 


=  di  J  ^  duih[x\[i.),[£,,m\rl,),,.i<q.3  <r) 

■/;(rJ(«)H:r:(u)).2;(u))rfu„(u)).  (58) 

It  can  be  shown  that,  bj-  using  the  averaging  method  of  Theorem  31' 
we  can  replace  the  G{  )  and  g{  )  in  (5.6)  by  G(-)  and  a(')  without  changing 
the  limits  as  (  —  0  and  T  oc.  Only  a  few  of  the  details  will  be  given. 
Consider  the  term  (5.9)  which  is  a  component  of  the  third  line  of  (5  8) 

f  (2-,' {t'))(^o(i, '(«).*, '(«^))«ft^  (5  9) 

Let  s  =  for  an  integer  n  and  /  >  0.  and  rewrite  the  integral  as 

rAT\(u))Go{TW.z\{M))du  (5  10) 

Dcf.iu  liic  shifted  rescaled  processes  ijo(ti)  =  z' [i  •>-  fu).r,\  =  r'(t  —  cn) 
.Now  lake  one  of  the  summands  in  (5.10)  and  change  the  time  scale  s  —  a 
tc  get 

I  W»+iS+<)/r 
‘  )/» 

V»e  study  (5.11)  as  —  0  and  c/^  —  0.  Note  that  for  each  >  C. 

SU7  P(sup  !r((u,)  -  i((0)'  >  ^t.,)  —  0  ^5  11) 

i<cx  ,</ 

as  —  C 

Define  the  occu^'aiior.  measures  P'’^'*(.)  and  fC’'*"'  ty 

f  X  B )  =  i.  / 


'  .'c 


ii‘.T.  c<  i  i }  equa.s 


^  J  fr<o{0))Go<o[0).i[0))P;)\{d<:>di}. 

By  an  ‘■occu7'ation  measure  argument"  similar  tc  that  used  with  Thee- 
rem  3.2.  and  using  the  fact  that  r,'*,^,,(-)  vanes  arbitrarily  little  over  ine 
interval  [C .  (  as  implied  by  (5. 12)).  we  can  show  that  &s  ('<  —  C  and 


f  —  0.  the  difference  between  )  and  the  measure  whose  a--componcnt 

IS  concentrated  on  )  and  whose  2»component  is  concentrated  on 

the  siationarj  fixed-z  2o(  )  process  with  parameter  converges 

weakly  to  zero.  This  convergence  ts  uniform  in  (i.r).  Thus 


£’:7  /  /; (^;o(t^))Go(x;o('^). 

^  J {■r  )  r 


u.'iTorjii.v  j;; 

of  (5.8)  by 


~.i  Ttius  we  car,  replace  the  ngh:  hand  i>ar;  of  tii<.  tlorc  iin 


{v))GiT*{u) .  ci)A, »/( do  )du. 


Similarly  for  the  0(2,  r)  term  in  the  fourth  line  of  (5  8) 

Analogously  tc  the  case  of  Tneorem  2.2,  the  right  hand  side  of  (5  8! 
goes  to  zero  in  probability  as  e  —  C  and  T  —  oc.  We  can  also  replace  the 
!:(•)  in  the  cost  function  by  !-(•)  without  changing  the  limits  Thus  the  cost 
functional  has  the  same  limits  as  those  of 


i-(C'(s ).  o  fo,  {da  )]diPj  (dpdi'  1 

The  rest  of  the  proof  is  no  different  than  that  used  for  Theorem  2.2  and 
the  details  are  omitted  Q.E.D. 

We  define  a  f  —  ojUmcl  ftcdhcck  control  j  or  the  citrcced  fisitn.  tc  h 
cry  micc^nrcHc  {'•xalntd  feedback  coni’ll!  for  which  the  ctfruccc’ 

:f.C,  ha}  c  unictt  Utah  stnst  solution  and  a  unique  tnicricnt  rneesuri  and 
such  that  for  anx  x.t'  and  admissible  I rr){-) .  v'{  )) .  r/f x  u^  )  <  *>  r'  m  u  — 
t.  lie  nou  stale  the  'ajj’roiimate  ojtimaliti  theorem’'. 

Theorem  5.2.  Assume  (Ad . ] j-fA5.S).  end  consider  the  ai  trace: 
/;  f  :  wuth  cost  function  ■^{x.rn.u  )  For  each  f  >  C.  let  then 
be  a  continuous  f  — optimal  feedback  control  u*'^  )  with  the  propeni  tf.a: 
{x’itj  x' [i ).  e  >  {..i  <  oz]  is  tight.  Then 

!>.r.u^Ui.u;)  —  j{u^)  <-:en(  (5  loi 

in  probability  uniforrrih  in  each  compact  (x.x)~set.  as  e  and  T  cc  tc  their 
limits  For  fl,;t  >  0  and  sequence  of  admissible  m'(  )  satisfying  fATs) 

hm  sup  P{ -4  (z.;.Tr/,u;.  Ur)  <  be  -  ^i]  =  C  (5  l-i) 
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Proof.  (5  13j  follows  by  Theorem  5.1,  with  the  replaced  by  u*(  ). 

If  (5.13)  IS  not  true,  then  there  is  a  >  0  such  that  the  limsup  is  greater 
than  6cj  Choose  a  weakly  convergent  subsequence  of  {Pt(  )}  ''•^h  the 
limit  measure  having  the  values  {■)  and  with  (f*  (•).  rri*' (•))  denoting  the 
induced  process.  Then 

:  7(m‘’,u‘')  <  -jc  -  ^i)  > 

a  contradiction  to  the  ^-optimality  of  t;*(  )  for  small  enough  f  Q.E.D. 


C.REFLECTED  DIFFUSIONS 

\V(  V,  ill  do  til-  hiialc'g  of  the  method  of  Section  3  for  a  refi'  cied  d.f.i-.  i. 
Tiie  ’  Skorohc'd  problem’'  model  of  the  reflected  diffusion  will  lie  used 
The  results  of  Sections  4  and  5  can  readily  be  extended  to  tins  case  We 
will  use  the  following  assumption. 

(AC  l)  r  i.‘  ilt<  clc-sv^  of  c  bounded  ojirn  sf1  tn  H'  will-  c  tv  ice  conltti- 
uousli  differrr.ttclU  boundarxcT.  Ui  n[x)  denote  the  outvard  norvie!  ti 
cF  at  X.  and  let  5i2-|  denote  the  reflection  direction.  Suppose  that  is 

the  rcstnciion  to  oF  oj  a  junction  which  is  twice  continuously  dtffereniiail'. 
in  c  neighhrhooc  oj  oF  and  lei  there  be  qi  >  0  such  that  ~3'{x]v[x'  >  o:. 
all  X  ^  dF 

The  Shorohod  problem  Let  u(  )  be  a  standard  vector-valued  J,  W'ler.er 
process,  W'e  say  that  x{  )  solves  the  Skorohod  problem  if  it  is  J, -adapted, 
continuous,  and  there  is  an  JF, -adapted  function  )  (•)  suck  that  for  ^  €  T. 


'■  f' 

x'j‘  =  x^  I  i',x{s))dsT  I  e{x[s)idu[S] -r  y  {i). 
vtc'V ■  = 'v  f '.  =  /  ,,.(ar)d ri,f  1. 


J: 


3  X  s  )'  ■  f  I 


Define  tne  si.. fie;  functicr.  -  -1  -  'j'yil  Defn-  t:.-:  oc¬ 

cupation  measure  P'  y  i  for  the  pau  of  processes  (x,  (■ ).  A,  V  i:.  ■  l  i .  and  then 
def.ne  Ft  as  in  (1.2).  In  order  to  get  the  needed  tightness  in  the  theo¬ 
rem  below  we  will  need  the  following  result  [11. Theorem  Let  C'iC.T' 

denote  the  space  of  continuous  /T-valued  functions  on  the  interval  \0-T_ 
with  the  sup  norm  topology. 


Theoreiri  C.l. 
Skorohod  proHerr, 


Assume  (AC.JJ  and.  for  each  T  <  oc  ccnsider  the 
xi,  =  t<Te.  (t.2) 
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uhrn  fi  )  and  t!  )  arc  trt  C’fC.T^  and 


lc[1}  ~ 

i^-KO  =  ^{s,,)){0T  )cfit|(5) 


Ij  f\-]  iJ  If;  a  compact  set,  then  (r(  ).  it(  ),  lll:|(  ))  arc  in  a  compact  set  in 
C*  ,V.J , 

Tiieoreni  C.2..4ssKmf  (Al.l)  and  ^A6.1).Then  {x,(-).A,|y|(.),<  < 
cic  )  and  {Pr{  \.T  <  oc)  art  itpht  Let  P{  )  denote  the  Itmit  of  c  ucakli 
conrcrpcTil  svtscguence  of  {Pj{).T  <  oc )  Then,  for  almost  all  ,  the 
ycii:rl'.  lulu-  /■'  )  tudhcts  c  statn-hnn.  proce  sf-  T"  <  ■ )  salr'-fvmc  :(  ]  Al'i 

Loldi- 

Proof.  Tilt  proof  similar  ic  that  of  Theorem  3.2.  The  lightness-  of 
{j-,;  I  e,  I'l  I  r  <  oc  ]  IS  a  consequence  of  Theorem  C.l.  and  liit  tigiitness 
of  the  .sequence  of  processes 


ij  h[T,lv)}dv  J  rlz,(v))du,(v).s 


<  ocf 


The  lightness  of  the  sequence  of  measure-valued  random  variables  follows 
from  me  iig.Mness  of  ihe  above  set  of  processes  Let  P[  )  denote  the  hm,;  of 
a  weakly  convergent  subsequence  of  {P7-(-).  T  <  oc ) ,  and  let  (r**  (■).  'V-  t'-)! 
aenoie  the  process  induced  by  the  value  P^{-).  The  and  V-ly)  are 
continuous  processes  and  x“(t  )  €  F  for  all  1.  Also  IV"!!’)  car.  increacse  only 
wher  f“(t|  €  oT .  The  stationarity  of  the  limit  processes  is  proved  as 
Theorem  3  1 

We  need  only  characterize  the  limits  x“(-).  Let  p'-)  denote  the  ger.er.c 
pat;,  .r,  T'  ’’i  m  as-r-ciated  w;::.  the  p-rcress  r''(  ).  and  y'-i  tiis  g'.r.e:,;  ;.a‘:. 
ir,  D'  [c.  oc  :  assoc. ated  with  the  p-rocess  )  !'  |  Ivede''.nt  the  function  Fi,  i 

used  beiov.  ,'3  3,'  as  fohows 

)  -  h  e  :,  t  y  .  -  Se  f  o  -  -  s; ,  -  ;  -  o  - 1 -- 

I  A/i'o.  u )  icu  —  I  (p.  L  )  J.  o-^u)  ic’y  u  J 

Tnis  function  is  def.ned  for  all  c.  )  €  F’iO  oc)  and  for  all  y'-l  ir.  T>' [C .  oc  j 
un.cr.  are  of  oounaed  variation  If  y'  )  is  no;  of  bounded  variation,  set 
tne  value  of  me  function  equal  ic  some  very  large  value  Define  P.”  ~ 
’y"  M-*). £  M  Then 

I  Af[r^v})du 

Jc 
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IS  an  J’j-marungale  This  implies  that  there  is  an  u  -  (•)  such  that  the  triple 
>■“  (  j.u*' (  Jj  satisfies  the  Skorohod  problem  (t  1)  QX,D, 
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